Decomposing generation networks with structure prediction for recipe generation

• We divide the recipes into phases, and use a global structure prediction component to assign different subgenerators generate recipe phases. incorporate attention mechanism get phase-aware features, which are input for produce better Our proposed framework outperforms previous state-of-the-art results. Recipe generation from food images ingredients is challenging task, requires interpretation of information another modality. Different image captioning where captions usually have one sentence, cooking instructions contain multiple sentences obvious structures. To help model capture avoid missing some details, we propose novel framework: Decomposing Generation Networks (DGN) with prediction, more structured complete outputs. Specifically, split each instruction several sub-generators phase. approach includes two ideas: (i) learning structures (ii) producing phases in sub-generator output based on predicted structure. Extensive experiments large-scale Recipe1M dataset validate effectiveness our model, improves performance over